The present invention relates to intruding object monitoring techniques, and more particularly to an intruding object monitoring method and an intruding object monitoring system capable of monitoring a plurality of intruding objects with a plurality of monitoring apparatus.
In a recent intruding object monitoring system, some intruding object monitoring apparatus using an image pickup device such as a camera as an image input means does not rely upon a manual monitoring method by a guard man, but automatically detects an intruding object in a monitor area, controls a pan/tilt head of the camera to always capture the intruding object in the view field of the camera in accordance with the detected position of the intruding object, and changes the camera direction and zoom setting to issue a predetermined notice or alarm, to record the image of the intruding object, and the like.
A monitoring apparatus, particularly an apparatus using two cameras, is disclosed in JP-A-5-334572 and JP-A-6-153051.
In the description of the specification, xe2x80x9ctrackingxe2x80x9d and xe2x80x9cfollowingxe2x80x9d are defined as in the following.
xe2x80x9cTrackingxe2x80x9d: A process of monitoring by detecting or measuring the positional change of an object on an image taken in the view field of an image pickup device such as a monitor camera.
xe2x80x9cFollowingxe2x80x9d: A process of monitoring an object by controlling the pan/tilt head of an image pickup device (that is, controlling of the direction of the camera optical axis).
First, in order to describe problems associated with a conventional intruding object monitoring system, a method of monitoring a tank yard with two types of TV cameras such as shown in FIG. 8 will be described.
Referring to FIG. 8, a first camera 802 is an image input means for sequentially picking up an image in the whole monitor area 804 at predetermined times. A first monitoring means including this first camera 802 detects an intruding object 801 from a plurality of frame images picked up at different times, and supplies the detection information of the intruding object 801 to a second camera 803. The second camera 803 is an image input means for automatically performing following of the intruding object 801 and sequentially taking the images thereof at predetermined times, in accordance with the detection information of the intruding object 801 supplied from the first camera 802.
Referring again to FIG. 8, by using a plurality of frame images taken with the first camera 802 at different times, differences between luminance values of each pixel are calculated and an area having a large difference is detected as the intruding object. When the intruding object 801 is detected, the first camera 802 controls the pan/tilt head of the second camera 803 in accordance with the detected position of the intruding object. In this manner, the intruding object 801 can be captured in the view field 805 of the second camera 803.
FIG. 9 is a diagram illustrating the principle of an intruding object detecting method to be used by the first camera 802. This method called a frame subtraction method has been used widely.
Referring to FIG. 9, a subtractor 909 calculates the difference between luminance values of each pixel in an input image 901 taken at time t0xe2x88x921 and an input image 902 taken at time t0, and outputs a difference image 904 between the input images 901 and 902. Similarly, a subtractor 910 calculates the difference between luminance values of each pixel in the input image 902 taken at time t0 and an input image 903 taken at time t0+1 and outputs a difference image 905 between the input images 902 and 903.
Next, a binarizing unit 911 generates a binarized image 906 by setting xe2x80x9c0xe2x80x9d if the luminance value of each pixel in the difference image 904 is smaller than a predetermined threshold value Th, and, for example, xe2x80x9c255xe2x80x9d (assuming eight bits of the luminance value of each pixel) if the pixel luminance value is equal to or larger than the threshold value Th. Similarly, a binarizing unit 912 generates a binarized image 907 from the difference image 905 by the method described above.
Next, a logical product unit 913 calculates a logical product between the luminance values of each pixel in the binarized images 906 and 907 and outputs a logical product image 908. With the above-described operations, human-like objects 914, 915 and 916 in the input images 901, 902 and 903 are calculated as difference areas 917 and 918 by the subtractors 909 and 910. The binarizing units 911 and 912 derives images 919 and 920 as clusters of luminance values xe2x80x9c255xe2x80x9d. The logical product unit 913 detects an image 921 as a cluster of luminance values xe2x80x9c255xe2x80x9d of each pixel in both the images 919 and 920. In this manner, the image 921 is detected as an intruding object.
Instead of the frame subtraction method, other methods may also be used if they can detect an intruding object in a monitor area.
As described above, when the first camera 802 detects an intruding object 801, the first camera 802 supplies control signals to the second camera 803 in accordance with the size and position of the intruding object 801 detected with the first camera 801. In response to the control signals, the direction (pan/tilt head direction) of a monitor view field 805 of the second camera 803 can be controlled and the intruding object 801 can be captured with the second camera 803.
As described above, the second camera 803 starts operating when an intruding object is detected from images taken with the first camera 802, and the zoom lens and taking direction (pan/tilt head direction) of the second camera 803 are controlled in accordance with the size and position of the intruding object detected from the images taken with the first camera 802.
A control amount of the zoom ratio of the second camera 803 is set in the following manner. For example, if the vertical image size of the first camera 802 is 240 pixels and the vertical image size of the detected intruding object is 30 pixels, then the zoom lens control amount of the second camera 803 is set to an eightfold (=240/30) of the zoom lens control amount of the first camera 802.
By setting the control amounts of the zoom lenses of the first and second cameras 802 and 803 in the above manner, the intruding object detected with the first camera 802 can be displayed over the whole screen of the second camera 803.
A method of calculating a control amount of the pan/tilt head of the second camera 803 will be described with reference to FIG. 10. FIG. 10 illustrates how a second camera 1003 performs following of an intruding object 1001 detected in a view field 1004 of a first camera 1002. For the purposes of simplicity, in FIG. 10, the first and second cameras 1002 and 1003 are set at the same position, and the center direction (optical axis direction) of the view field of the first camera 1002 is made coincident with a reference view field direction (pan/tilt head control angles: pan 0xc2x0 and tilt 0xc2x0) of the second camera 1003, paying attention only to the x-axis direction.
In FIG. 10, 2 W represents the number of pixels (unit: pix) in the horizontal (x-axis) direction of an input image in the whole view field (monitor area) 1004 of the first camera 1002, xcex94x represents a displacement (unit: pix) in the horizontal direction of a detected intruding object from the view field center of the first camera 1002, xcex8w represents a half monitor angle (unit:xc2x0) of the first camera 1002, and xcex8x represents a pan control angle (unit:xc2x0) of the pan/tilt angle head of the second camera 1003. A half monitor angle xcex8w of the first camera 1002 is given by:                               θ          ⁢                      xe2x80x83                    ⁢          w                =                              tan                          -              1                                ⁡                      (                                          1                2                            xc3x97                              h                f                                      )                                              Eq. (1)            
where h (unit: mm) represents the horizontal length of the image pickup device of the first camera, and f (unit: mm) represents a focal length of the first camera.
Under this monitor conditions, the number 2 W of pixels in the horizontal direction of an input image, the displacement xcex94x in the horizontal direction of a detected position of an intruding object from the view field center of the first camera 1002, the half monitor angle xcex8w of the first camera 1002, and the pan control angle xcex8x of the pan/tilt angle head of the second camera 1003, have the following relation:                                           tan            ⁢                          xe2x80x83                        ⁢            θ            ⁢                          xe2x80x83                        ⁢            x                                tan            ⁢                          xe2x80x83                        ⁢            θ            ⁢                          xe2x80x83                        ⁢            w                          =                              Δ            ⁢                          xe2x80x83                        ⁢            x                    W                                    Eq. (2)            
Namely, the pan control angle xcex8x of the pan/tilt angle of the second camera 1003 is given by:                               θ          ⁢                      xe2x80x83                    ⁢          x                =                              tan                          -              1                                ⁡                      (                          tan              ⁢                              xe2x80x83                            ⁢              θ              ⁢                              xe2x80x83                            ⁢              w              xc3x97                                                Δ                  ⁢                                      xe2x80x83                                    ⁢                  x                                W                                      )                                              Eq. (3)            
For example, if the half monitor angle xcex8w is 30xc2x0, the number 2 W of pixels in the horizontal direction of an input image of the first camera 1002 is 320 pixels, and the displacement xcex94x in the horizontal direction of a detected intruding object from the view field center of the first camera 1002 is 40 pixels, then the pan control angle xcex8x of the second camera 1003 is 8.2xc2x0.
As described above, a plurality of frame images taken with the first camera 1002 at different times are compared to calculate differences between luminance values of each pixel. When an area having a large difference is detected as the intruding object 1001, the zoom lens and pan/tilt head of the second camera 1003 are controlled in response to control signals which are supplied from the first camera 1002 in accordance with the size and position of the detected intruding object 1001. Since the intruding object 1001 can be captured always in the view field 1005 of the second camera 1003, the second camera 1003 can automatically perform following of the moving intruding object 1003.
If the conventional two-camera structure is applied to such a monitoring system, the following problem occurs. Since a plurality of second cameras for performing following of a plurality of intruding objects are provided, the process amount of the first camera increases in proportion to the number of intruding objects to a problem, because the first camera is required to control the plurality of second cameras.
Further, if the first camera cannot detect temporarily an intruding object, it is not possible for the second camera to continuously perform following of the object, even if the object is at the position where the second camera can capture it.
Furthermore, an intruding object can be detected only if it is in the monitor area of the first camera. Therefore, if the monitor area of the first camera is set so that an intruding object can be taken in a size capable of being detected reliably, the monitor area is reduced.
For example, if an intruding object having a horizontal width of 1.0 m or wider is required to be detected as a cluster having a width of ten pixels or more of a binarized image and the image size is 320xc3x97240 pixels, then the monitor area of the first camera has the horizontal length of only 32 m (=1.0/(10/320)). In this case, the area outside of the monitor area of the first camera becomes a blind area while an intruding object is subjected to following by controlling the zoom lens and pan/tilt head.
If an intruding object enters an area behind a building in the monitor area or the blind area caused by an operation limit of the pan/tilt head, the intruding object cannot be subjected to following continuously.
It is an object of the present invention to provide an intruding object monitoring method and an intruding object monitoring system capable of performing following of a plurality of intruding objects by distributing or decentralizing the load of a monitoring process.
It is another object of the present invention to provide an intruding object monitoring method and an intruding object monitoring system capable of performing following of an intruding object with high reliability.
In order to achieve the above objects, according to one aspect of the present invention, there is provided an intruding object monitoring method using an intruding object monitoring system having a first intruding object monitoring apparatus for monitoring a whole monitor area by sequentially inputting images of the area and at least one second intruding object monitoring apparatus for performing following of an intruding object, wherein the first intruding object monitoring apparatus acquires detection information of an object intruding into the monitor area, the second intruding object monitoring apparatus captures the intruding object in accordance with the detection information supplied from the first intruding object monitoring apparatus. After the intruding object is captured, the second intruding object monitoring apparatus performs following of the captured intruding object independently of the first intruding object monitoring apparatus.
In one embodiment, when tracking of the intruding object by the first intruding object monitoring apparatus becomes impossible, the first intruding object monitoring apparatus continues the tracking by acquiring detection information of the intruding object from the second intruding object monitoring apparatus. Accordingly even if the intruding object is missed, tracking of the intruding object can be continued.
In another embodiment, when the following of the intruding object by the second intruding object monitoring apparatus becomes impossible, the second intruding object monitoring apparatus continues the following by acquiring the detection information of the intruding object from the first intruding object monitoring apparatus. Accordingly even if the intruding object is missed, the following of the intruding object can be continued.
According to another aspect of the present invention, there is provided an intruding object monitoring system comprising: a first intruding object monitoring apparatus for sequentially inputting images of a monitor area and acquiring detection information of an intruding object in the monitor area; at least one second intruding object monitoring apparatus for capturing the intruding object by controlling a pan/tilt head in accordance with the detection information acquired by the first intruding object monitoring apparatus, and performing a following of the captured intruding object, wherein the first intruding object monitoring apparatus includes an intruding object monitoring unit, the second intruding object monitoring apparatus includes an intruding object monitoring unit separately from the intruding object monitoring unit of the first intruding object monitoring apparatus, and the second intruding object monitoring apparatus performs the following of the intruding object detected by the first intruding object monitoring apparatus, independently of the first intruding object monitoring apparatus.
In an embodiment, the system has a plurality second intruding object monitoring apparatuses, the intruding object monitoring unit of the first intruding object monitoring apparatus judges whether a new intruding object exists, and if it is judged that a new intruding object exists, identifies one of the second intruding object monitoring apparatuses now not in following, and the identified second intruding object monitoring apparatus captures the newly detected intruding object and performs the following.
It is preferable that the second intruding object monitoring apparatus nearer to the position of the intruding object is assigned the following of the intruding object or that when a plurality of intruding objects are detected, the second intruding object monitoring apparatus is assigned the following of the intruding object existing in the monitor area having a highest priority order.
According to another aspect of the present invention, there is provided an intruding object monitoring apparatus comprising: a camera having a zoom lens; an image processing unit for detecting an intruding object from image signals supplied from the camera; a pan/tilt head of the camera; a pan/tilt controller; a zoom controller; a communication unit for communication with at least another intruding object monitoring apparatus; a display for displaying an intruding object monitoring result; and a control unit for controlling to switch among a plurality of monitoring modes of the intruding object monitoring apparatus and the other intruding object monitoring apparatus(es).
In an embodiment, if the monitoring mode of the intruding object monitoring apparatus is a normal mode (i.e. detecting mode) for monitoring a predetermined monitor area and if the intruding object monitoring apparatus receives a following take-over request from an adjacent other intruding object monitoring apparatus which is tracing an intruding object, the monitoring mode of the intruding object monitoring apparatus is switched to a following mode to take over and perform the following of the intruding object. Accordingly, even if the intruding object enters the blind area of the camera of one intruding object monitoring apparatus, another intruding object monitoring apparatus can take over the following of the intruding object to continue it.
According to another embodiment, if a view field magnification is requested from an adjacent other intruding object monitoring apparatus, the monitoring mode of the intruding object monitoring apparatus is switched to a wide angle mode and monitors an area which is broader than the monitor area in a normal mode and includes at least a partial area of the monitor area in the normal mode of the adjacent other intruding object monitoring apparatus. Accordingly, when an adjacent other intruding object monitoring apparatus enters the following mode, the intruding object monitoring apparatus can monitor its own monitor area and cover the monitor area of the adjacent other intruding object monitoring apparatus.
The foregoing and other objects, features and advantages of the invention will become apparent from the following more particular description of the embodiments of the invention as illustrated in the accompanying drawings.